FAST HYBRID INTERPOLATION METHODS 



2 BACKGROUND OF THE INVENTION 

3 1 . Field of the Invention 

4 This invention relates to fast hybrid interpolation methods that provide high quality images. The 

5 hybrid interpolation methods employ more than one interpolation algorithm and choose the 

6 most appropriate algorithm that provides high quality images with a minimum processing time. 

c:J 2. Description of the Related Art 

=jp Interpolation is a basic operation in image processing and is routinely used in image 

B 9 enlargement and reduction. For instance, interpolation is used in image and video resizing. It is 

$f) also required in single chip CCDs that are widely used in digital cameras and camcorders. 

H Furthermore, interpolation is used in sub-pixel motion estimation in video compression and is 

|1> also required in de -interlacing. 

1 3 Numerous interpolation algorithms have been proposed, and there are significant differences in 

1 4 complexity and performance. Usually, a simple and fast interpolation algorithm requires a small 

1 5 number of operations but provides a relatively limited performance. On the other hand, a 

16 complex algorithm requires a large number of operations but provides a much better 

1 7 performance. For instance, the nearest neighbor interpolation algorithm is the simplest and 

1 8 fastest. However, the resulting image is somewhat blocky. Although the bilinear interpolation 
1 9 algorithm is fast and provides relatively good performance, it tends to blur edges. High degree 

20 spline interpolation methods provide much better results, but the processing time increases 

21 significantly due to the large increase in operations. In addition, there are many other 

22 interpolation algorithms that do not use polynomials. For example, interpolation algorithms 
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1 using neural networks have also been proposed. Although they provide relatively good 

2 performances, they require a large number of operations due to the nature of neural networks. 

3 In general, there is a large variation in numbers of operations among various interpolation 

4 algorithms. Typically, interpolation for two-dimensional images is performed as a series of 

5 one-dimensional interpolations. In other words, rows are processed first, and then columns are 

6 processed, or vice versa. When interpolation is performed separately in each direction, the 

7 value of a new pixel is obtained from its two neighbor pixels in the linear interpolation, 

8 requiring 2 additions and one multiplication. In the cubic spline interpolation, the value of a 

9 new pixel is computed from its four neighbor pixels, requiring 3 additions and 4 
10 multiplications. Thus the processing time increases roughly by a factor of 4 when the number 
1HS of multiplications is considered. If the B-spIine of degree 5 is used for interpolation, the value 
1B5 of a new pixel is computed from its 6 neighbor pixels, requiring 5 additions and 6 
13j multiplications. Although one can obtain a better quality image if a high degree B-spline 
14f interpolation algorithm is used, the high degree B-spline interpolation requires a significantly 
1 j§ longer processing time. In other words, one obtains a better quality image if a complex 
14| interpolation algorithm is used. However, such a complex algorithm requires a large number of 

17 operations, thus significantly increasing the processing time. 
fU 

1 8 In many applications, the processing time is critical. For instance, the single chip color filter 

19 array CCD, which is widely used in digital cameras and camcorders, needs interpolation 

20 operations to fill missing colors in the color filter array (Fig. 19). If the processing time is too 

21 long, this can cause a serious problem. For a digital camera, it will take a long time to take a 

22 picture and store it. For a camcorder, a fast processor may be required since it should handle at 

23 least 30 images per second in order to produce acceptable video signals. Furthermore, when 

24 interpolation is used for video resizing or de-interlacing, the processing time is also critical. 

25 Otherwise, the processor may not display the video properly. If interpolation is used for sub- 

26 pixel motion estimation in video compression, the processing time is very critical. Thus, if one 
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1 can devise an interpolation algorithm that is fast while providing high quality images, such an 

2 algorithm would have many useful applications. 

3 In general, a complex interpolation algorithm that requires more computations provides a better 

4 performance than a simple interpolation algorithm that requires fewer computations. However, 

5 in many cases, the differences are negligible in most pixels. Major differences occur around 

6 edges where high frequency components are dominant. Thus, if one can predict whether the 

7 difference will be small or large, one can use a simple interpolation algorithm for regions where 

8 differences are small and employ a complex interpolation algorithm for regions where 

9 differences are large. In other words, prediction means is first applied to determine which 
1 0 interpolation algorithm is the most appropriate algorithm that provides high quality images with 
1j| a minimum processing time. Then, one can obtain high quality images without significantly 
1r| increasing the processing time since in most images the difference between the simple 
t| interpolation algorithm and the complex interpolation algorithm would be minor except in some 
1 J small regions. 

1ig It is noted that the prediction means, which predicts whether the difference will be small or 

1 S lar S e > should be simple and fast. Otherwise, the saving in processing time may not be so large. 

W Thus, according to the teaching of the present invention, prediction means is first applied. 

1 8 Then, a simple but fast interpolation algorithm is used for pixels where the simple interpolation 

1 9 algorithms provide acceptable performances and a complex interpolation algorithm is used for 

20 pixels where the complex interpolation algorithm, which requires a larger number of 

21 operations, significantly outperforms the simple interpolation algorithm 
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1 SUMMARY OF THE INVENTION 

2 There is a great need for a fast interpolation algorithm that provides high quality images. 

3 Generally, a complex interpolation method requiring more computations provides a better 

4 performance than a simple interpolation method that needs fewer computations. By noting that 

5 the differences between the simple and the complex interpolation algorithms are minor in most 

6 regions with major differences occurring around edges where high frequency components are 

7 dominant, the present invention provides hybrid interpolation methods which employ more 

8 than one interpolation algorithm and choose the most appropriate interpolation algorithm that 

9 provides high quality images with a minimum processing time. 



Thus, it is an object of the present invention to provide a hybrid interpolation method that is fast 
while maintaining high quality images. 
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1 BRIEF DESCRIPTION OF THE DRAWINGS 

2 Fig. la shows the B -spline of degree 0. 

3 Fig. lb shows the B-spline of degree 1. 

4 Fig. lc shows the B-spline of degree 3. 

5 Fig. 2a illustrates how a new pixel value is computed using the linear interpolation (B-spline of 

6 degree 1). 

7 Fig. 2b illustrates how the linear interpolation (B-spline of degree 1) works for 1-dimensional 

8 data. 

9 Fig. 3a shows a one-dimensional data. 

1 0 Fig. 3b shows the coefficients of the B-spline of degree 3 of the data of Fig. 3a. 

1 1 Fig. 3c illustrates how the continuous model is obtained from the coefficients of the B-spline of 

12 degree 3. 

1 3 Fig. 3d illustrates how the cubic spline interpolation (B-spline of degree 3) works. 

1 4 Fig. 4 shows an original image. 

1 5 Fig. 5 shows an image enlarged by a factor of -Jl using the bilinear interpolation algorithm. 
1 6 Fig. 6 shows an image enlarged by a factor of *$2 using the cubic spline interpolation 

1-7 algorithm. 

1 8 Fig. 7 shows the difference image between the two images enlarged by the bilinear and cubic 
1 9 spline interpolation algorithms. 

20 Fig. 8 shows the histogram of the difference image. 

21 Fig. 9 shows the table of the differences between the bilinear and the cubic spline interpolation 

22 methods along with proportions and accumulations. 

23 Fig. 10 shows the flowchart of a hybrid interpolation method that employs two interpolation 

24 algorithms. 

25 Fig. 11a shows the flowchart of a hybrid interpolation method that chooses one of two 

26 interpolation algorithms, depending on the difference between the two nearest neighbor 

27 pixels. 

28 Fig. 1 lb shows how the difference between the two nearest neighbor pixels is computed. 
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Fig. 12a shows an interpolation example. 

Fig. 12b illustrates how the hybrid interpolation method of Fig. 11 works. 

Fig. 13 shows the flowchart of a hybrid interpolation method that employs more than two 

interpolation algorithms. 
Fig. 14 shows the flowchart of another hybrid interpolation method that employs more than 

two interpolation algorithms. 
Fig. 15a shows an interpolation example. 

Fig. 15b shows a case in which the bilinear interpolation algorithm provides a good 
performance. 

Fig. 16 shows the flowchart of a hybrid interpolation algorithm that chooses one of the 

interpolation algorithms depending on the slopes of the adjacent pixels. 
Fig. 17 shows a more efficient implementation of the hybrid interpolation of Fig. 16. 
Fig. 18a shows an interpolation example. 

Fig. 18b shows a case in which the bilinear interpolation algorithm provides a noticeably 
different interpolation result from that of the cubic spline interpolation algorithm. 
Fig. 19 shows an example of the Bayer color filter array. 
Fig. 20 shows the green color pattern in the Bayer color filter array. 
Fig. 21 shows the red color pattern in the Bayer color filter array. 

Fig. 22a shows the flowchart of a hybrid interpolation that chooses one of the interpolation 
algorithms depending on the differences between four adjacent pixels. 

Fig. 22b shows how the differences between four adjacent pixels are computed. 

Fig. 23 shows a more efficient implementation of the hybrid interpolation of Fig. 22 when 
interpolation is used for a color filter array. 

Fig. 24a shows the 4 nearest neighbor pixels of a new pixel to be interpolated. 

Fig. 24b shows the 16 nearest neighbor pixels of a new pixel to be interpolated. 

Fig. 25 shows a more efficient implementation of the hybrid interpolation of Fig. 17. 
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DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS 



2 Embodiment 1 



3 The standard procedure for interpolation is to fit the digital data with a continuous model and 

4 then resample the function at a desired sampling rate. Quite often, splines, which are piecewise 

5 polynomials, are used in interpolation. In particular, B-splines are favored in image 

6 interpolations. 

7 In the B-spline interpolation model [1], the signal is modeled as follows: 
| s(x) = ^c(j)P"(x-j) 

$ where P"(x) is the B-spline of degree n, and c(j) is the B-spline coefficient. In particular, 

1 ij ft (x) and P 3 (x) are given by 



P\x) = 



2/3- | x | 2 +|x| 3 12 0<|x|<l 
(2-|x|) 3 /6 l<|x|<2. (2) 

0 2 <| x | 



13 Fig. la shows the B-spline of degree 0, p\x). Fig. lb shows the B-spline of degree 1, 

14 /^(x). Fig. lc shows the B-spline of degree 3, P\x). For splines of degree 0 (the nearest 

15 neighbor interpolation) and splines of degree 1 (the bilinear interpolation), the B-spline 

1 6 coefficients, c(k), are the same as the signal samples, s(k). In other words, 

17 c(k) = s(k). 

1 8 For high-degree B-splines, the coefficients can be computed using a digital filter as follows [1] : 

19 c(k) = Wrs(k) (3) 
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1 where b"(k) = /3"(x)|^. For the cubic spline interpolation, the coefficients can be computed 

2 using the following filter: 

3 ftVwA r . (4) 

z+4+z 1 

4 where <-» represents the z-transform. 

5 Typically, interpolation for two-dimensional images is performed as a series of one- 

6 dimensional interpolations. In other words, rows are processed first, and then columns are 

7 processed, or vice versa. Similarly, three-dimensional interpolation can be performed as a 

8 series of one-dimensional interpolations. When interpolation is performed separately in each 
;& direction, the linear interpolation computes the value of a new pixel from its two neighbor 

I |j pixels. The cubic spline interpolation computes the value of a new pixel from its four neighbor 

pixels. 

12" Thus, for the bilinear interpolation (Fig. 2a), the value of a new pixel at P 0 120 will be 

1 jfjj calculated as follows: 

I I y = ax[2] + bx[l] = ax[2] + (l-a)x[l] = x[l] + a(x[2]-x[l]) (5) 

1 |ij where x[l] and x[2] are signal samples. As can be seen in the above equation, two additions 

1 6 and one multiplication are required to compute the value of the new pixel using the bilinear 

1 7 interpolation algorithm. Fig. 2b shows the result when the bilinear interpolation method is 
1 8 applied to 1 -dimensional data. 

1 9 As stated previously, for the cubic spline interpolation, the B-spline coefficients first need to be 

20 computed. In order to compute the B-spline coefficients, two additions and two multiplications 

21 are needed for each pixel as can be seen from Equation (4). Fig. 3 illustrates the procedure of 

22 the cubic spline interpolation method. From the digital data (Fig. 3a), the B-spline coefficients 

23 130 (bold line) are computed (Fig. 3b). The interpolation result is shown in Fig. 3c. Then, the 

24 value of a new pixel will be calculated from the coefficients of its four neighbor pixels using 

25 Equation (2). For instance, the value of a new pixel is computed as follows: 
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1 y = c 0 /3 3 (l + a) + Cl l3\a) + c 2 P\l-a) + c 3 p\2-a), (6) 

2 as illustrated in Fig. 3d. Thus, 3 additions and 4 multiplications are required to compute the 

3 value of the new pixel using the cubic spline interpolation algorithms, assuming that J3 3 (l + a), 

4 p 3 (a) , fi 3 (2- a) and - a) are already computed and saved. Since multiplication is one of 

5 the most time-consuming operations in hardware implementation, it can be said that the 

6 processing time of the cubic spline interpolation is about 4 times that of the bilinear 

7 interpolation without counting the computation time for the B-spline coefficient. It is noted that 

8 there are many other methods for implementing the high degree spline interpolation methods 

9 that include cubic spline interpolation methods. It is further noted that the present invention is 
1 0 not restricted to any special implementation of the spline interpolation methods. 

Next, it will be illustrated how two interpolation algorithms can be used to produce a high 

1@ quality image without significantly increasing the processing time. Fig. 4 shows an original 

13j image. Fig. 5 shows the image enlarged by a factor of V2 using the bilinear interpolation 

14 algorithm and Fig. 6 shows the image enlarged by the same factor using the cubic spline 

1 S5 interpolation algorithm. Fig. 7 shows the difference image between the two images enlarged by 

1 £ the tw0 inter Polation algorithms. As can be seen in Fig. 7, most pixels of the difference image 

1 !?j are very small in value, indicating that the performances of the two interpolation algorithms are 

1 8 almost identical except in some small regions, particularly around edges. Fig. 8 shows the 

1 9 histogram of the difference image of Fig. 7. Fig. 9 shows the table of the differences between 

20 the bilinear interpolation and the cubic spline interpolation algorithms along with proportions 

21 and accumulations. As can be seen in the table, for about 67% of the whole pixels, there is no 

22 difference between the bilinear interpolation and the cubic spline interpolation algorithms. 

23 Furthermore, for more than 96% of pixels, the differences are smaller than 5. Typically, when 

24 the difference is small enough, the human eye cannot distinguish between them. Only about 4% 

25 of the pixels have differences larger than 5. The large differences occur along edges where 

26 pixel values change abruptly. 
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As stated previously, the processing time of the cubic spline interpolation algorithm is roughly 
4 times that of the bilinear interpolation. Thus, if it is possible to devise means to predict 
whether the difference between the bilinear interpolation and the cubic spline interpolation 
algorithms will be small or large, one can use the bilinear interpolation for pixels where the two 
interpolation algorithms provide similar performances and the cubic spline interpolation 
algorithm for pixels where the cubic spline interpolation algorithm significantly outperforms the 
bilinear interpolation algorithm. Then, high quality images can be obtained without increasing 
the processing time significantly. For example, if the differences between the bilinear and cubic 
spline interpolation algorithms are larger than a threshold for only 5% of the total pixels, one 
can use the bilinear interpolation algorithm for 95% of the total pixels and the cubic spline 
interpolation algorithm for the remaining 5%. In this case, the processing time increases by 
about 15% compared to the bilinear interpolation algorithm, assuming that the processing time 
is proportional to the number of multiplications. If the cubic spline interpolation is used for the 
whole image, the processing time will increase by roughly 300%. Thus, one can obtain image 
qualities that are comparable to those of the cubic spline interpolation algorithm, while the 
processing time is comparable to that of the bilinear interpolation. Although the bilinear 
interpolation and the cubic spline interpolation algorithms are used in the above example, it is 
noted that any two interpolation algorithms that differ in performance and complexity can be 
combined. 

It is noted that the prediction means, which predicts whether the difference between the bilinear 
interpolation and the cubic spline interpolation algorithms will be small or large, should be 
simple and fast. If the prediction means is complex and requires a large number of operations, 
the saving in processing time may not be as large as one may wish. Thus, according to the 
teaching of the present invention, prediction means is first applied. Then, a simple but fast 
interpolation algorithm is used for pixels where it provides acceptable performance, and a 
complex interpolation algorithm is used for pixels where the complex interpolation algorithm, 
which requires a larger number of operations, significantly outperforms the simple interpolation 
algorithm. Fig. 10 illustrates the hybrid interpolation algorithm. 
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1 When interpolation is performed as a series of one-dimensional operations, a possible test is to 

2 check the two pixels that are nearest to the pixel to be interpolated. If they are similar in value, 

3 one can use a simple interpolation method that requires a small number of operations. If the 

4 difference between the two pixels is larger than a threshold t, one can use a complex 

5 interpolation method that usually requires a large number of operations. In other words, by 

6 examining the two pixels that are nearest to the pixel to be interpolated, one can determine what 

7 kind of interpolation method should be used. The flowchart of this procedure is shown in Fig. 

8 1 la. In order to find a new pixel value at x new 210 (Fig. 1 lb), the two nearest neighbor pixels 

9 x[j] 211 and x[j + l] 212 are examined. If the difference 213 between the two pixels is 
1 ;Q smaller than a threshold t, a simple interpolation is used. Otherwise, a complex interpolation 
13[ algorithm is used. 

1 3 Fi S- 12 illustrates how the procedure of Fig 1 1 works when the bilinear interpolation algorithm 

13 and the cubic spline interpolation algorithm are used. In order to find a new pixel value at P l 

% 220, the two nearest neighbor pixels x[l] 222 and x[2] 223, are examined. Since the 

1g difference is smaller than a threshold which is assumed to be 5, the bilinear interpolation 

1 jS; algorithm is used, producing a value 225. Next, in order to find a new pixel value at P 2 221 , 

1 7 the two nearest neighbor pixels x[2] 223 and x[3] 224 are examined. Since the difference is 

1 8 larger than the threshold, the cubic interpolation algorithm is used, producing a value 226. The 
1 9 detailed implementation of the cubic spline interpolation algorithm can be found in [1]. In the 

20 above example, the bilinear interpolation and the cubic spline interpolation methods are used. 

21 However, one can combine any two B-spline functions with different degrees. For instance, 

22 one may use the bilinear interpolations and the B-spline of degree 7. In fact, one can combine 

23 any interpolation methods that differ in performance and processing time to produce an image 

24 quality that is comparable to that of the complex interpolation algorithm with a processing time 

25 that is comparable to that of the simple interpolation algorithm. 
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1 Numerous interpolation algorithms have been proposed by many researchers. Each 

2 interpolation algorithm has advantages and disadvantages. The idea and teaching of the present 

3 invention can be further extended to hybrid interpolations that employ more than two 

4 interpolation algorithms. With the proper prediction means, one can choose the most 

5 appropriate algorithm that provides high quality images with a minimum processing time. 

6 It is noted that there are many other tests which can be used to choose the most appropriate 

7 interpolation algorithm that provides high quality images with a minimum processing time. It is 

8 also noted that instead of a single prediction means, a series of prediction means can be used to 

9 choose the most appropriate interpolation algorithm, particularly when more than two 
10 interpolation algorithms are employed. For instance, in Fig. 13 a first prediction means is 
1J a PP lied t0 each P ixel to be interpolated. If a condition is met, Interpolation Algorithm 1 is used. 
1 J Otherwise, a second prediction means is applied. According to the outcome of the second 
1 3j prediction means, either Interpolation Algorithm 2 or Interpolation Algorithm 3 is used. Fig. 14 
ti4j shows another prediction means when three interpolation algorithms are used. Depending on 
1 5 the out come of the prediction means, one of the three interpolation algorithms is selected. 

1J| Embodiment 2 

1 7 One of the primary regions where the difference between a simple interpolation method and a 

1 8 complex interpolation method is large is a region where changes of pixel values are irregular. 

1 9 Consider the example of Fig. 15a, where the numbers 255 represent the pixel value. In Fig. 

20 15a, the slopes between adjacent pixels from xfj-1] 251 to x[j + 2] 254 are relatively 

21 uniform. In this case, the bilinear interpolation method will provide relatively good 

22 performance for a new pixel at p 250, although the difference between the two nearest 

23 neighbor pixels, x[j] 252 and x[j + l] 253, is large, as can be seen in Fig. 15b where the 

24 solid line 256 represents the bilinear interpolation and the dotted line 257 the cubic spline 

25 interpolation. As can be seen, the interpolation values at p are almost identical. Therefore, 

26 another possible prediction means is to check the slopes of the adjacent pixels. If the slopes are 
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similar, a simple linear interpolation algorithm is used. If the slopes vary significantly, a 
complex interpolation algorithm, such as the cubic spline interpolation algorithm, is used. Fig. 
16 shows the flowchart of this prediction means. In Fig. 16, a new pixel at p is to be 
interpolated. First, the largest integer, j, that does not exceed p is computed by the following 
operation: 

j = int(p) . 

Then, the difference between x[j + 1] and x[j] and the difference between x[j] and x[j - 1] 
are compared. If the following conditions are not satisfied: 

(x[j + 1] - x[j])(x[j] - x[j - 1]) > 0 AND 
\x[j + l] + x[j-l]-2x[j] \<t 
where t is a threshold, a complex interpolation algorithm is used. Otherwise, the difference 
between x[j + 2] and x[j + 1] and the difference between x[j + 1] and x[j] are compared. If 
the following conditions are satisfied: 

(x[j + 2] - x[j + l])(x[j + 1] - x[j]) > 0 AND 
\x[j + 2] + x[j]-2x[j + l]\<t, 
then a simple interpolation algorithm is used. Otherwise, a complex interpolation algorithm is 
used. 

The location of the next pixel to be interpolated is determined as follows: 
p = p + A, 

where A is the inverse of the scale factor. In other words, if the image is enlarged by 2, A is 
0.5. If the image is to be enlarged, A will be always smaller than 1. In this case, one can use 
the prediction mechanism of Fig. 17, which reduces the number of comparisons. In particular, 
the image is to be enlarged by a factor of 2, the prediction means in Fig. 25 can be used, which 
is more efficient. 

Fig. 18a shows an example in which the variance of slopes in the vicinity of a new pixel P 4 
280 is large. In this case, a complex interpolation algorithm, such as the cubic spline 
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interpolation, might provide a better performance than a simple interpolation such as the bilinear 
interpolation. For instance, the pixel value P cu 281 that the cubic spline interpolation provides 
appears to provide a more accurate interpolation result than the pixel value P bi 282 that the 
bilinear interpolation algorithm provides (Fig. 18b). 

Embodiment 3 

In order to reduce cost and size, the single chip CCD is widely used instead of 3 CCD sensors 
in consumer electronics such as digital cameras and camcorders. In the single chip CCD, color 
images are obtained by a color filter array (CFA). Fig. 19 shows an example of the Bayer color 
filter array. In Fig. 19, G stands for green, R for red and B for blue. In order to obtain a full 
color image, the missing colors should be interpolated. In practice, it is important to obtain a 
good quality image using an interpolation algorithm that does not require too many operations 
for single chip CCD sensors in consumer electronics. Otherwise, it would increase cost and 
size. Furthermore, interpolation methods that require a large number of operations tend to 
consume a large amount of power, which is a limited resource in portable electronics such as 
camcorders or digital still cameras. The idea and teaching of the present invention can also be 
applied to this case. In other words, in order to interpolate a missing color, the prediction 
means is first applied to determine which interpolation algorithm is most appropriate. Then, a 
simple but fast interpolation algorithm is used for the missing color where the simple 
interpolation algorithms provide acceptable performances and a complex interpolation algorithm 
is used for the missing color where the complex interpolation algorithm, which requires a larger 
number of operations, significantly outperforms the simple interpolation algorithm. 

There are many possible ways to fill in those missing colors. For example, in order to fill a 
missing green color in Fig. 20, one may interpolate the missing value from its vertical neighbor 
pixels. Alternatively, one may interpolate the missing value from its horizontal neighbor pixels. 
Another possible way is to take the average of the interpolated values that are obtained from the 
horizontal and vertical neighbor pixels. In order to fill in a missing red color in Fig. 21, one 
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1 may first fill some of missing pixels from horizontal neighbor pixels and then fill some of 

2 missing pixels from vertical neighbor pixels. The remaining missing pixels can be filled in the 

3 same way as the green color. It is noted that there are many other possibilities to fill in the 

4 missing colors. The teaching of the present invention is not concerned with the order in which 

5 the interpolation is performed. 

6 Embodiment 5 

7 The example of Fig. 15 shows that if the rates of change in the adjacent pixels are relatively 

8 uniform, the simple bilinear interpolation method provides a good performance even though the 

9 difference of the two nearest pixels (*[;] 252, x[j + 1] 253) is large. On the other hand, Fig. 
1g 18 illustrates a different case. If the rates of change in the adjacent pixels have a large variance, 
1 2 the bilinear interpolation algorithm provides a very different interpolation result from that of the 
t2| cubic spline interpolation algorithm, even though the difference between the two nearest pixels 
T3j (x[j] 283, + 284) is small. Fig. 22a shows another possible prediction means that 
1j| takes this phenomenon into account. In order to interpolate a new pixel value at p 320 (Fig. 
1ft 22b), the four nearest neighbor pixels (x[j - 1] 325, x[j] 324, x[j + 1] 326, x[j + 2] 327) 
1 li are examined with index j which is the largest integer that does not exceed p. Then, a simple 

1 7 interpolation algorithm is used only if all three differences ( D, 321 , D 2 322, D 3 323) of the 

1 8 four neighbor pixels (x[j - 1] 325, x[j] 324, x[j + 1] 326, x[j + 2] 327) are smaller than a 

1 9 threshold (Fig. 18b). In the case of the single chip color filter array where the color patterns are 

20 very regular, the prediction scheme in Fig. 23, which is more efficient, can be used. An 

21 advantage of the prediction means in Fig. 23 over that of Fig. 22a is that the prediction means 

22 in Fig. 23 needs a smaller number of comparisons. It is noted that the method in Fig. 23 can 

23 also be used if the image is to be enlarged by a factor of 2. 



24 Embodiment 6 



1/30/02 - 17 of 20- LEE 

When the bilinear interpolation method and the cubic spline interpolation method are employed, 
the B-spline coefficients first need to be computed. As stated previously, in order to compute 
the coefficients of the B-spline of degree 3, 2 multiplications and 2 additions are required for 
each pixel. Although the cubic spline interpolation is used only for a small portion of the image, 
the B-spline coefficients are computed for the whole image. Thus, according to the teaching of 
the present invention, the cardinal spline of degree 3 is used instead of the B-spline of degree 3 . 
An advantage of the use of the cardinal spline is that one does not have to compute the 
coefficients, since they are the same as the signal values. However, a problem with the cardinal 
splines is that they have an infinite duration. In fact, the cardinal spline of degree n is given by 

r 1 n {x)=j j (b:r\k)p n {x-k) 

whose duration is infinite. One way to avoid this infinity problem is to apply truncation to the 
cardinal splines. Although truncation introduces some error, the error can be made arbitrarily 
small by making the truncation window large. Thus, if the number of pixels that need a high 
degree B-spline interpolation algorithm is small, the use of the cardinal spline may reduce the 
total processing time since the coefficients need not be computed. However, if the number of 
pixels that need a high degree B-spline interpolation algorithm is large, the conventional 
implementation will be more efficient. 



